- by John Luxford
Earth From Up Here is a news-style comedy show produced in Flipside that features an alien newscaster named Zēblō Gonzor, who sits at a UFO-shaped desk, with a screen behind him for showing photos and video clips. On his desk are a number of props - a coffee cup, a paper and pencil - and some buttons that trigger the show's intro, outro, and alien laugh tracks. The concept is similar to Saturday Night Live's Weekend Update and Last Week Tonight with John Oliver.
Earth From Up Here was designed for a two-person team made up of a producer/director and a writer/performer. Here is how we make an episode of the show.
The producer/director searches for funny news stories and videos online and adds them to a preparation document.
The writer/performer takes this document and writes a script based on the materials provided.
Based on the script, the writer and producer also compile the needed photos and video clips. We use Photoshop and Adobe Premiere for this step.
We then upload the images and video files to the web. We use Backblaze B2 to store these files, but you could also use Amazon S3 or any website or file hosting service that provides public links to the files.
Now we're ready to launch Flipside and enter the studio. Once inside, we select the Earth From Up Here set.
To add our slideshow links, we right-click the Slideshow button under the Activities tab and edit the slideshow settings. We paste the links in one-per-line in the order they need to appear for the show.
Next, we do the same thing for the Teleprompter and paste the script into its settings, so the talent can read from the script while they're performing (Figure 5).
Now that the show is ready to be recorded, we also launch OBS Studio to record the video output. Flipside controls OBS Studio transparently using the obs-websocket plugin. This allows Flipside to focus on recording the VR version of the show, while OBS Studio simultaneously handles recording the video as well as the ability to stream a video feed live to Twitch, YouTube, Facebook Live, and other streaming services.
To ensure OBS Studio is ready, we click the Connect button and wait for the OBS Status to say Connected.
When we click Record, the show starts and the actor performs while the director cuts camera angles. Either the actor or the director can control the teleprompter and slideshow, depending on what works best for the team members.
When a take is good, the director or the actor can click the star icon on that take to keep track of the good ones for later.
So that's the process of producing an episode of Earth From Up Here in Flipside. As you can see, Flipside streamlines much of the time it takes to produce an animated show, and enables the talent to focus on what they do best.
- by John Luxford
By Lauren Cochrane, Improviser & Writer - Bucko
Welcome to part 2 of our blog post series on working with live actors in VR (click here to read part 1). This part is about working as an actor in VR, and is a guest post courtesy of our friends at Bucko (aka Genefur and 2B from Super Secret Science Island).
Improv is and always will be about listening. Connecting to your scene partner requires communication both verbally and physically. In VR improv, the same is true, but you will need to listen in a whole new way.
You won’t have the same physical connection as you would in a live theatre setting, so it’s imperative that you take your time and speak and move with intention. It’s actually quite a gift for an improvisor to have the opportunity to play in a format that slows you down and makes you come at a scene from a completely new perspective.
Yes, you will absolutely react to and communicate with the character’s movements (both your own, and your scene partner) but the movements themselves are different. For example: If I am in regular live scene, and I put my hand to my ear to listen to something in the distance, that’s a pretty simple gesture that I can do to communicate an offer and I know my scene partner will follow along. In VR, my character’s body and limitations are different than my own.
Same scenario as above: Genefur’s ear is about half a foot away from my head because the character’s head is wider than mine. So I, as the actor, have to think about how to move my body and puppet Genefur to put her hand to her ear, to communicate the same offer of listening to something in the distance.
This adds a few extra seconds to my offer. Both scene partners have to be aware of this added time. If you are just talking and flailing around you will miss things and the audience will won’t be able to immerse themselves in the world. So, it takes a little bit to get used to the timing and new reaction process. Practice and play.
It’s important to be aware of your virtual surroundings and space, so you don’t bump into things you can’t feel. Using the monitors inside the VR scene is really important. You will want to get familiar with the environments and props to know where they end and you begin.
Yes, it can be funny to use the world and hide inside it (like when Genefur hid inside the fridge to scare 2B) but it can take away the reality if you walk through the table or each other when you’re not supposed to be able to - just like in live improv - if I set up the scene and indicated that “this is where the grand piano is, it’s big and beautiful and right there” and then I walk through it, it dispels the reality I worked so hard to create. Same goes with VR. It will take the viewer out of the magic and then the “oops” becomes the focus, rather than the scene.
Trust is still conveyed between virtual avatars, even though you’re acting against the other person's character and not their real self. If you are playing with someone who you are used to playing with, it will make the VR scene process MUCH easier. It’s a little funny to get used to, but knowing that the person you are playing with is right there with you in the scene, while also at the same time, being in a room down the hall-it’s kind of new sensation.
Your brain will honestly believe you can reach out and physically touch your partner. Aaron and I usually high five to connect and ground ourselves together in live scenes. And without talking about it, we naturally did that between takes in VR scenes.
When you come out of VR and realize this person your brain swore was beside you isn’t, it’s a very cool/weird/eerie/amazing feeling. It’s a feeling that is probably easiest to process, if you are super comfortable with your scene partner. It’s a new level of trust. As an improvisor that is quite the experience!
Lack of an audience can take a little bit of adjustment. Live improv is pretty clear with it’s audience communication. You, as the improvisor will know if the audience is digging what you are putting out by their laughter, clapping - even just the vibe of the room. You can feel and see if the audience is enjoying themselves and connecting.
In VR, you won’t have that, which can take a little bit of getting used to. You have to trust the same instincts and timing like you would in a live theatre, but be okay without the instant payoff of a room laughing and giggling with you. You’re a seasoned improviser. You got this. Trust it’s working, and you can go back and try again if it’s not.
In live improv, you will constantly be told “Cheat to the audience!” They need to be able to see what you are doing, what you are holding (mimed or real) and what you are trying to do with it. It’s the absolute same in VR improv.
The cool thing about props in the VR scenes is that they are used to add even more to the world than you could ever have in live scenes. (Ahem: Super Science Pencil-YES!) So if you use or make something, be sure to check the monitors to see that it’s purpose is being communicated.
That if you drew a sword and are now using it, make sure the camera’s and your scene partner can see it clearly. Otherwise, you will have a break in communication. If that happens, scroll up and re-read this again. It’s all there my friends. :)
- by John Luxford
By John Luxford, CTO & Co-founder - Flipside
With Flipside’s first two show productions in full swing, we've now been through a number of production days with live actors working inside the platform. We learned a ton as a result, and wanted to share what we learned with you.
This first post explores some of the more general lessons we learned that have helped streamline our productions and helped us empower our actors to do the best job they can.
Processes need to be honed, but they also need to be documented. These are living documents that will evolve rapidly, but you won’t be able to iterate on them as fast if you don’t have them written down to begin with.
We currently have documentation covering:
These act as checklists to make sure we don’t miss a step that may have cost us time, or worse, a usable end result.
Because Flipside is still in beta, our "known issues and workarounds” document becomes critical. The purpose of this is to provide quick actionables we can use when an issue arises, without having to worry about finding a creative solution on the spot. Not having a workaround ready can quickly eat into your production time.
At first, there were little things that we would have to reset between takes, and early on some of these even required restarting the app, which doesn’t help the actors stay in the right frame of mind. Context switching hurts creativity, and our goal isn’t just to be a virtual studio, but to use that opportunity to eliminate as much of the friction that goes into show production as possible.
So we iterated on ways of reducing the time between takes as much as possible. We now have a process that is impressively automated, with one person manning the camera switcher and director tools, while the actors are free to concentrate their attention on what they do best.
Actors need to learn and get into their characters, how they move, how they talk. They also need to get comfortable acting with a VR headset on. One request we got was a simple mirror scene, so the actors could practice their parts while seeing themselves from the front, side profile, and back all at the same time. Actors can now hop in there and see exactly how their movements translate to their virtual counterparts.
The actors need to know when something is about to change, or where they should be standing and facing to be in frame. For example, we added virtual marks for each camera position, which update prior to the next camera change so the actors can know to move into place or turn if needed for the next shot.
This can also be as simple as counting down to “Action!” in VR when the director clicks record instead of starting recording immediately after pressing the button. These little things add up to make a more intuitive experience for everyone.
This means we put a lot of work into making eye contact feel right, and blinking feel natural, because we don’t have eye tracking available in consumer VR headsets just yet.
We also devised our own system for more natural neck and spine movement, as well as arms that emulate more traditional animation techniques that emphasize for style over accuracy. Since today’s full body inverse kinematics options still don't feel quite natural, and the closer you get to the character feeling alive, the more you risk falling into the uncanny valley.
The more you play into the strengths of the medium, the more the quality of the content can shine. Counter-intuitively, the better things get, the more noticeable the remaining issues become.
We quickly realized that even with lip syncing and natural eye movements, the avatar faces felt dead. To solve this, we created an expression system that the actors can control with the joystick on their motion controllers that allows them to express four distinct emotions, but also blend between them (smoothly transitioning between happy to upset, while blending naturally with the lip syncing).
With a little practice, these expressions can become reflexive actions for the actors, giving them a new level of expressive control as they embody their characters.
There are lots of unsolved problems in VR, probably most notably locomotion without causing motion sickness. But there are other subtler causes of motion sickness too, which can include anything that creates even slight disorientation.
Image source: techradar.com
One of the strangest examples we encountered in Flipside was in our preview monitors (which are just flat virtual screens for the actors to see the 2D output). We found that there was a perceived parallax in the preview monitors which caused a tiny amount of motion sickness over time. Nothing crazy, but present nonetheless. The solution we came up with was to flip the video on the preview screens horizontally. This had the effect of making any on-screen text appear backwards, but eliminated the perceived parallax which slowly caused discomfort for the actors.
The reason this is so critical is that actors are likely spending prolonged periods of time in the virtual sets, doing several takes before they get it just right, or doing batches of episodes in a single shoot. Anything that causes discomfort can potentially cut your shoot short in an unpleasant way.
These are some of the more general lessons we took away from working hands-on with live actors in a virtual world. They’ve helped us hone our vision, and Flipside is already way better because of it.
Stay tuned for the next post in this 3-part series on live acting in virtual reality. We have lots more to share! And if you're a content creator, make sure to sign up for early access to Flipside Studio!
- by John Luxford
By John Luxford, CTO & Co-founder - Flipside
Storytelling in virtual reality is about directing attention and engaging via interaction with the user. In film, there is the frame to direct attention, but in VR the user can look away when you didn't intend for them to and miss a key plot point or instruction.
Here, I'd like to explore how Teller's secrets that magicians use to manipulate audiences can be applied to build stronger narratives in VR.
Note: This post uses examples from common VR experiences, so it may contain spoilers if you haven't played them. I'll do my best to keep the examples simpler and relatively spoiler-free.
Repetition builds expectation, which will draw the user's attention back to the repeating action. For example in Accounting, the repeated use of putting on a virtual headset to move to the next part of the narrative, or the miniature lawyers that keep popping out of the briefcase in the courtroom scene.
If we look to create common metaphors and repetition around choices the user is given, it helps build familiarity and understanding of their surroundings so they can get lost in the details of the experience itself as they progress through it.
By making something more elaborate, we can string them along multiple steps, or wow them with the perceived complexity of the task, even if it involves just two or three elements. In the extreme example, a Rube Goldberg machine comes to mind, whose complexity can easily wow audiences both in and out of VR. Many simpler examples exist in VR due to the perceived physical nature of the medium.
In terms of immersion, this can be as simple as confirming a single interaction via multiple senses (visually via depth cues and believable movement, accurate spatialized audio, and haptic feedback to touch), which tricks the mind into believing the interaction was with a real object. This can lend believability to any interaction, from pressing a button to mimicking the feel of shooting a bow and arrow.
Probably the simplest but best example of this is a technique that originated in Job Simulator, which is that as soon as you pick up an object your virtual hand disappears and is replaced by the object itself.
Owlchemy Labs had discovered that when you're busy pouring virtual hot sauce on things and laughing at the hot sauce coming out of the virtual bottle, you don't even notice your hand isn't wrapped around the bottle anymore. This has the benefit of avoiding having to make the virtual hand look right when wrapped around a wide variety of objects with varying shapes, which when done wrong can highlight the limitations of the experience and break the sense of immersion as a result.
As an interesting aside, we opted not to use this technique in Flipside after much testing, because the perception of the viewer is more important than the perception of the actor in a show, and viewers found it odd that the actors' hands kept disappearing.
The frame in both magic on stage and story in VR is where the user's attention has been drawn. Fortunately in VR, the trickery is often in the form of making something appear or disappear, which can happen in a variety of ways.
You can make something appear right in front of the user to create an element of surprise, or worse you can make a monster appear beside them when they're not looking to create a jump scare.
To make a character disappear when they're no longer needed, they might walk out of a door or behind an obstruction and can be turned off when safely out of view. Meanwhile the user's attention is directed towards the next part of the experience.
This also relates to the example of combining multiple senses I used earlier, since much of VR's ability to immerse comes from the reinforcement of information across multiple sensory inputs. The eye can't be wrong if the ear heard it too.
But in terms of narrative and progression, combining two tasks or actions can be a great way to reinforce the direction of a user's attention, helping ensure they're seeing the key points along the way.
For example in Rick and Morty: Virtual Rick-ality, the need to use the microverse machine to complete the task of recharging the batteries adds a second layer of interaction to the original goal, which immerses the user in a deeper way.
Bringing physicality into the experience by requiring the user to carry out a task itself is a way of combining the story element with participation. When the user is busy doing, we can be busy setting up the next steps for them.
To use something from Flipside in an example, we have a VR episode of Super Secret Science Island in our not-yet-released Flipside viewer app where 2B throws a magic pencil out to the audience, breaking the fourth wall. If the viewer catches the magic pencil, the magic pencil grows to match the size of the viewer's hand. Most users we've tested don't even notice it happen, which means they've suspended their disbelief.
In fact, the magical action of throwing something out of the pre-recorded show and into your hand, akin to a character throwing something out of the television and into your living room, doesn't even seem that far fetched as it happens. The pencil appears in your hand as expected, and most users simply begin to draw with it.
Whenever the user can inspect an object for themselves, they'll believe what their senses tell them.
The simplest example here isn't exclusive to VR games, but any sandbox narrative style game offers the user a limited set of choices, and as you make these choices you begin to feel that the game's outcomes are tailored to your individual set of choices. But in reality there are only a handful of final outcomes in the game's sandboxed world.
Some VR examples include Push for Emor, the previously mentioned Rick and Morty: Virtual Rick-ality or the highly-anticipated Fallout 4 VR.
As you can see, there are many parallels between VR narratives and traditional forms of live entertainment like theatre, improv, and magic. Magicians especially have had millenia to hone the craft of manipulating perception, which makes for a rich body of work to learn from as we craft our narratives in this new medium. Once again, what is old becomes new again, which might be the oldest trick in the book.
- by John Luxford
Today is Demo Day for Tribe 9 at Boost VC, and we are super excited to be a part of it and proud of the work everyone has done over the past few months. The growth and polish in everyone's products, pitches, and businesses is incredible.
The past week alone has been crazy for us. We announced Flipside Studio in early access as well as the first two shows created with it, Earth From Up Here and Super Secret Science Island. We demoed like mad with Exit Reality at VRLA, and had some great coverage in Tom's Hardware, VR Status, and The VR Base.
Anyway, we better get back to rehearsing and getting our setup looking spiffy for the event. Looking forward to showing off the future of animation and interactive content creation in VR!
- by John Luxford
We’ve been hinting at big things for a while now at The Campfire Union, and we’re super excited to finally unveil the first part of those things. Flipside has been our semi-stealth mode project since early last year, and we’re now ready to start opening it up to a wider audience.
Today, we are making Flipside Studio available to a select group of content creators. We need your feedback to shape it into the best product it can be, and it’s time to see what you guys come up with. If you are a content creator and are interested in participating, please sign up here:
Flipside Studio transforms your standard PC-based VR setup into a complete virtual TV studio, where you can create live-acted animated shows in real time. Your headset and controllers are the motion capture system as you embody characters on your own virtual sets.
Shows are recorded for both 2D screens as well as VR and AR playback. 2D shows can be viewed on any of today’s screens, and can even be streamed live to Facebook, Twitch, or YouTube.
Today we’re also proud to share the first show made in Flipside Studio, created in collaboration with Winnipeg-based improv comedy duo Bucko Comedy. Super Secret Science Island is an improv comedy show about two failed science experiments, 2B and Genefur, who are stranded on a deserted island. Follow their antics as they look for a way off the island and to find their missing creator, Dr. Whoosh.
As if one show wasn’t enough, we’re also launching with a weekly alien news update about what’s happening down here on Earth. The alien newscaster, named Zeeblow Gonzoar, is written and performed by San Francisco-based comedian Jordan Cerminara.
We will be sharing lots more info about the making of Super Secret Science Island and Earth From Up Here, including advice and tips on acting in VR courtesy of Lauren and Aaron of Bucko Comedy, in the weeks to come.
If you are a content creator, streamer, writer, actor, producer, or director and would like early access to Flipside Studio for a show of your own, enter your email below to be added to our early access list:
Stay tuned for more Flipside productions in the weeks to come! We’ve only just begun to discover what’s possible in terms of finding new ways of interacting with audiences, and of bringing the impossible to life.
Sincerely,
The Campfire Union
Join the Community
Join the Community
Quest
Rift